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ABSTRACT 

Resource allocations, in terms of funds, people, 
facilities, and the delegation of appropriate authority to formulate 
appropriate policy, for research and development and implementation 
of computer-assisted instruction are discussed in this paper. A 
description and justification of CAI as a technology is included. The 
need for incorporating a. systems approach to educational innovation 
is stressed. A partnership among industry (profit and nonprofit), 
government, and education is suggested as a model, and a national 
network of multidisciplinary centers is advocated as the vehicle for 
accomplishing the goals of research, development, and implementation 
of effective and efficient CAI systems. A reference list is appended. 
(JY) 
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Prefatory Note 
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Virginia, under Work Unit IMPACT, Prototypes of Computerized Training 
for Army Personnel. 

The paper was presented by Dr. Seidel at a conference titled 
“Application of Computers to Training” sponsored by the National Security 
Industrial Association (NSIA) in February 1970. 

An earlier version of this paper was presented by Dr. Seidel at the 
National Council for Educational Technology (NCET), Computer Based 
Learning Seminar, University of Leeds, Leeds, England, September 1969. 



RESOURCE ALLOCATIONS TO EFFECT OPERATIONALLY USEFUL CAI 

Robert J. Seidel and Felix F. Kopstein 

INTRODUCTION 

The broad picture of resource allocation for research and development 
(R§D) and implementation of computer- administered (or computer-assisted) 
instruction (CAI) will be discussed in this paper. Specific cost 
projections within an operational setting have been discussed previous 1/ 
(Kopstein and Seidel, 1) and therefore will be dealt with only briefly. 
"Resource allocation" will refer to funds, people, facilities, and the 
delegation of appropriate authority to formulate appropriate policy. 

And the fact that all these many resources— government, industry, and 
education will be considered— means that inevitably politics and 
economics will be intertwined in the subsequent discussion. 

The features of this paper will include a brief description and 
justification for CAI as a technology. The need for incorporating a 
systems approach to educational innovation will follow to set the 
proper framework for the magnitude and complexity of the required 
research and development effort. An analogue to a corporate level 
of investment in research and development, 3% to 5% of income, is 
proposed to effect the orderly transition of CAI from "breadboard," 
through prototype, to an operational system. 

Problems that arise in the course of considering appropriate resource 
allocation stem in part from the fact that none of the so-called enti- 
ties — government, industry, or education — are monolithic. Government 
involvement will be both central and local. Industry in CAI includes 
hardware manufacturers, book companies, and so forth. Education 
includes central and regional administrators, teachers, research and 
development personnel, as well as the ultimate user, the student-. 

Problems of program management and coordination are thereby made 
difficult. 

Finally, a partnership model will be proposed to evolve operational 
CAI. The vehicle of a national Rf*D center with regional satellites is 
suggested both to provide proper training and to permit flexibility of 
research approaches to accomplish the goal of operational CAI. These 
points are discussed in the context that: "anything worth doing is 

worth doing right." 

Thus, we will ask if research and development into CAI is worth 
expenditure of money, time, and effort. As you might predict, our 
answer will be affirmative. Granted this is so, what expenditures are 
necessary to "do it right"? Finally, how should the resources be 
properly allocated among industry, government, and the educational 
community to bring CAI to fruition? Fruition in this case refers to 
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the actual production of multiple copies of operational CAI systems 
usable in schools and universities throughout a nation. 



THE FRAMEWORK 



CAI— A Necessary Technology 

Various prominent individuals and agencies in the United States 
have indicated the need and desirability of making use of the educa- 
tional opportunities inherent in individualized instruction. The 
Committee for Economic Development has represented the broadest cross- 
section of U.S. society in its appraisal (200 businessmen, educators, 
psychologists, and community leaders). In essence, they present a 
convincing case for a failure of our current educational system to take 
advantage of modern technology and to deal effectively with the increas- 
ing requirements of our complex society. The Committee has pointed 
out that "individualized instruction geared to the individual interests, 
abilities and learning rate is one of the cherished goals of American 
education. It is an aspiration which we wholeheartedly share, yet the 
schools are making very slow headway in this direction through present 
means ..." (2) . Psychologists have been engaged in the study of 
individual differences for years. Teachers have long been complaining 
of the inability— administratively — of coping with students as indi- 
viduals and have been thereby forced to teach, for the most part, to 
the mean of a class. 

Our view is that CAI is the leading, operationally defined edge of 
a model of individualized instruction. It represents the potential 
(with all due respect to objection by Oettinger and Marks (3)) for a 
quantum leap in adapting instruction to the momentary needs and capa- 
bilities of the individual student. It provides the basis for an 
Iteratively improving instructional environment. 

A word should be added here about the distinction between technique 
and technology. The criteria for evaluating them differ. One technique 
may be properly compared to another within the same system model. 

Teaching reading by the phonics method versus "look and say" is an 
example of such a comparison Indeed, studies have been conducted 
attempting to hold all other components of the momentary education 
system constant save the difference in technique. 

In evaluating a new technology, however, such relatively clear and 
simplistic comparisons are insufficient. It may well happen, as with 
CAI and individualized instruction in general, that the system in 
question must be redefined. It is silly to mold a computerized, 
individualized teaching environment around a six-hour day. It is also 
improper to consider the economics of the new technology in terms that 
neglect the new opportunities opened up and hitherto impossible. To 
quote Harley, "When economies result [from using the new technology] 
they are derived from the reduced cost of providing additional services-- 
from the improvement in the quality of teaching and the level of learning; 
from the ability to shrink time and space, and from the sharing of limited 
resources," (4, p. 56). 
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CAI is thereby a technology, not a new experimental classroom 
technique. Indeed, the magnitude of societal effect possible with 
CAI may be compared without any hesitation, to that of the industrial 
revolution. The potential for basic improvements in educational 
systems exists, but it must be brought to the proper climax. The 
economic exploitation, frustration, disruption of employment, suicide, 
and so forth, which the industrial revolution led to, could have been 
avoided with proper planning. With the advent of the computer, automa- 
tion has already changed man’s role to that of innovator rather than 
routine performer. On a purely rational basis, therefore, CAI is 
worth doing wherever individual differences make a difference in the 
field of instruction. 

CAI Research a nd Development is Complex 

Having answered, all too briefly, our first question affirmatively— 
that is, CAI is indeed worth doing— the second and more thorny question 
must be addressed. What do we mean by, "It should be done right"? 

Figure 1 shows Eight Steps that comprise a total systems approach. This 
framework (with slightly different words, or with seven versus eight 
steDs) must be followed from research through development and implementa 
tion in order to result in operationally valid and useful CAI systems. 



The Eight Steps of the Systems Cycle 

1. State the real NEED you are trying to satisfy. 

2. Define the educational OBJECTIVES that will contribute to satisfying 

the real need. 

3. Define those "real world"’ limiting CONSTRAINTS that any proposed 
system must satisfy. 

4. Generate many different ALTERNATIVE systems. 

5. SELECT the best alternative(s) by careful analysis. 

6. IMPLEMENT the selected alternative(s) for testing. 

7. Perform a thorough EVALUATION of the experimental system. 

8. Based on experimental and "real world" results, FEEDBACK the required 
MODIFICATIONS and continue this cycle until the objectives have been 
attained. 



Figure 1 

A proper operational CAI system must pass from the breadboard 
stage, through the prototype, to an operational, cost/effective phase. 
"Breadboard" is a term originated by electronic circuit designers. S\ 

During very early stages of developing an electric circuit the paper 
design (conception) is translated into a set of components provisionally 
connected (by alligator clips and a few wires) and tacked onto a wooden 
board. The purpose is to verify that the design scheme will have the 
general characteristics expected of it. By extension, "breadboard" refers 
to any first and provisional realization of a system design. 
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‘Applied to CAI, documents like the CED (Committee for Economic 
Development, July 1968, 2) report have stated the need in education 
for making maximum use of individualized and personalized instruction. 

We have defined our objectives in terms of producing student output 
at a given level of achievement. As part of the design, note that 
this process is to be an iterative one in attaining that goal. In 
the breadboard iteration, only the most crucial design criteria are 
applied. Secondary objectives, that is, desirable features or H nice- 
to-have" characteristics, are kept from confusing the basic design 
problem. For example, with reference to CAI, at the breadboard stage 
it is inappropriate to consider time-sharing the computer with batch- 
processing operations. 

Similarly, during the breadboard i^tage of design, operating 
constraints are minimized. In terms of CAI, it may be essential to 
develop inexpensive student terminals, but first terminals with adequate 
characteristics must be designed. Further limitations stem from avail- 
able computers and compatible CAI equipment and languages. Current CAI 
systems are divided among those which have not progressed past the 
breadboard stage and those which have tried (unsuccessfully) to bypass 
this stage. Alternative instructional decision-making strategies and 
mixes of hardware and software subsystems with selected subject matter 
must be considered. Following systematic evaluation and study of the 
alternatives, selection of an initial system is made and tYnplemented 
for a test run. Evaluation is made based on student output. Feedback 
to improve (modify) iteratively the CAI system, namely, meeting objec- 
tives, is made. 

This process continues throughout the iterative development in 
order to refine the system. Once the breadboard phase has been com- 
pleted, it is possible to proceed to a prototype system design, the 
circumstances under which the system must perform. In this phase one 
first establishes precisely what the system is to do, taking into con- 
sideration major constraints such as permissible costs or delivery time. 
Various available means (e.g,, magnetic or optical information storage) 
are weighed against optimization criteria. Optimization means a oest 
compromise among contradictory objectives and imposed constraints 
(e.g., lightweight, portable student terminals with character-video- 
audio display capability for no more than $500 per unit) in terms of 
some ordered set of criteria (price more important than display 
capability, which is more important than portability). Finally, the 
design plan that has emerged is implemented and a first prototype is 
synthesized. A prototype CAI system may have operational usefulness, 
but is likely to include design flaws and oversights that ought not 
be multiplied in many duplicated installations. A prototype is merely 
an untried and unadjusted assemblage. 

A tested system emerges over a number of subsequent repetitive 
development cycles. In the case of a CAI system, only a small number 
of students would be exposed to its instruction initially. Their 
interactions with the system must be minutely monitored and appropriate 
adjustments made. Massive data need to be collected in order to 
determine whether the prototype is actually performing as envisioned. 
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Where actual and expected performance disagree (e.g., mean delay of 
system response to student exceeds stipulated value of one second) 
revisions must be made in the system design. A tested system exists 
only after observed system performance coincides with expected perform- 
ance. Only then is it economically justifiable to use the prototype 
design as a template for multiple reproductions. 

This approach is also necessary when attempting .improvements in a 
traditional instructional model. We shall return and enlarge upon 
this approach later when proposing a scheme for accomplishing proper 
CAI development. The significance of the approach is amplified ten- 
fold with an innovation as encompassing as CAI can be. The framework 
then does not preclude CAI as drill and practice, simulation, or use 
as strictly a calculational aid for problem solving. These can all 
be useful parts of an overall CAI R£D project. However, as previously 
noted, the instructional system consists of N components. If only one 
or two components are clearly identified', measured, and controlled, 
their contribution to instructional output relative to that of all 
the N-l or N-2 other uncontrolled components becomes extremely diffi- 
cult, if not impossible, to assess (5). 

"Doing it right" thus means having a careful system research and 
development effort integrating the facets of computer hardware and 
software with content development and studies of alternative instruc- 
tional strategies. This means, in turn, a multidisciplinary effort. 

If CAI is to fulfill the promise of delineating the relative importance 
of the various characteristics pertinent to appropriate decision-making 
in instruction, it must have the opportunity to model the instructional 
situation, to vary parameters, and to apply these to meaningful human 
learning. 

Tha Amount of Reqoi red Effort for CAI R&D is Large 

Let's consider what experience to date has shown to be the require- 
ments of large-scale CAI centers for annual funding levels. From 
informal discussions with knowledgeable persons regarding their CAI 
projects’ fiscal problems, it appears both internationally 1 and in the 
United States that an operating budget of approximately $250,000 to 
$300,000 a year is necessary merely to maintain facilities in operations. 
The reason for this becomes quite clear if one considers simply the 
rental price of an IBM 1500 system as an example. The hardware alone 
averages $100,000 to $110,000 per year. Taking the $250,000 total, this 
leaves $150,000 per year for staffing, administration, and support 
facilities. The result is that, given approximately $38,000 cost per 
professional man-year, one can have only a minimal CAI program (4 pro- 
fessionals) concentrating on operational activities. If research is to 
be conducted at a CAI center in a university or other academic institu- 
tion (where most CAI RdjD seems to be taking place), it must depend 



!Mr. Andre Kirchberger, Administrator, Center for Educational 
Research and Innovation, Organization for Economic Cooperation and 
Development (OECD), Paris, France. 
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inevitably upon part-time participation of faculty, students, and 
administrative personnel. Most of these individuals have other 
primary responsibilities demanding the major part of their energies 
and attention. 

How much, then, should a total systems approach toward the develop- 
ment of CAI require to produce an operational, useful example of 
individualized instructional models? We cannot answer this in absolute 
terms, but on the basis of experience with Project IMPACT'S 1 multi- 
disciplinary staff involving 18 professionals and seven others m 
supporting roles, a personnel budget of at least twice these amounts 
seems necessary. Our installation is unique in that it is funded, so 
far, by a single arm of the government. Of course, one of the pioblems 
for most of the CAI projects is that they are funded from multiple 
sources with different demands; and in order to satisfy the require- 
ments of the various funding agencies, the research and development is 
fragmented. Until recently, the awareness of funding at a "critical 
mass" level did not seem to exist. Recently, it seems that the U.S. 
Office of Education has taken steps to support fewer projects in CAI 
but at a higher level of funding. This seems to be a move in the right 
direction. The problem, however, in maintaining this critical level 
is not simply to permit existence of an operating environment, but 
rather to facilitate large-scale, integrated centers to study and 
arrive at both the adequate descriptions of learning processes and 
the necessary prescriptions for instructional development. 

A recently published figure from the Bureau of Research, U.S. 

Office of Education (6), indicates that they are providing an average 
of approximately $287,000 per project over 10 CAI projects : If other 

governmental agencies could support these scon& projects, without 
changing their goals and with a comparable amount of mon?y, then it 
would seem to be possible to proceed beyond the breadboard stage of 
CAI development. 

A point of clarification is worth noting with respect to encouraging 
flexibility and diversity in approaches. The advocacy of varying 
approaches to solving the strategy development and overall construction 
of useful CAI should not be in any way misunderstood as the support of 
multiple small-scale efforts. Furthermore, if a funded project is 
required to continually submit and resubmit a multiplicity of proposals, 
a large amount of time and effort will go into nonproductive work. A 
rational basis for selection of project proposals should be used at the 
outset, and then a reasonable amount of funding be provided over a 
period of at least five years. Without advocating that all the eggs 
be put in one basket, we are saying that the highest rate of progress 
is likely to result from a distribution of available funds to fewer 
"baskets" at sufficient levels to permit large-scale integrated efforts. 
On the other hand, funding a diversity of projects at very small levels 
will probably result in none of the efforts providing a full-fledged 
operationally, implementable CAI system for education. 



iHumRRO Work Unit IMPACT, Prototypes of Computerized Training for 
Army Personnel. 
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RESOLVING THE PROBLEMS OF NECESSITY, COMPLEXITY, AND MONEY 



An Analogue to the Corporate 
3# to 5% Investment^ Ts Advocated 

What expenditures should be invested in a CAI total systems effort? 

We can start by taking a cue from a corporate model and note that a 
number of sources (Duckworth of the United Kingdom, 7, as well as 
representatives of corporations in the United States) have indicated 
a 3% to 5% level of corporate income is appropriate for these purposes. 

In 1960, the amount spent in education was approximately .1% (Coombs, 8). 
Recently, it was estimated that in 1968 the entire educational enter- 
prise in the United States cost somewhere around $50 billion 1 . The 
figure for expenditures on educational research now reaches approximately 
$100 million, a .2% investment. Perhaps some would find cause for 
optimism in this increase. However, even with a large error factor, it 
appears "that the investment in educational research is only a fraction 
of 1% of the educational enterprise." In contrast to this, R§D invest- 
ment in the electrical communications industry in 1968 stood at 
approximately 3.4%. IBM reportedly invested $300 million, or roughly 
5%, in R§D of its reported $6 billion gross income for the same period. 

If we accept the 3% to 5% figure as necessary to sustain viable 
R§D in education, the annual dollar investment, given the $50 billion 
total year expenditure, should have been between $1.5 and $2.5 billion. 
Before you cringe at the apparent enormity of this amount, consider it 
relative to other innovative programs. The United States has put a man 
on the moon. The United States has invested approximately $24 billion 
and 10 years of research and development to go from the breadboard 
stage through the prototype to the ultimate system for the lunar 
landing and retrieval. Accepting the 3% to 5% figure, a comparable 
time period for innovative developments in education would amount to 
an investment between $15 and $25 billion. We need to consider this 
question: Is education any less valuable than a lunar landing or space 

travel in general? Our personal reaction is that both efforts are 
extremely valuable. 

Let us carry the analogy one step further. The dollar cost of 
investment in education, in particular in CAI, further pales by compari- 
son to the costs of other single technological developments. For 
example, the U.S. government’s contribution so far to the controversial 
supersonic transport program is roughly $1 billion. The cost of the 
deep-dive nuclear research submarine is roughly $99 million. Many more 
such examples could be cited. Note that these are specific system 
efforts. We have previously documented that an implementable computer- 
administered instructional system, following from breadboard through 
operational development stages, could be accomplished for something 



Arthur S. Melmed, "CAI— A Five-Year Perspective," a symposium 
presentation at American Educational Research Association meeting 
(mimeographed copy), Los Angeles, February 1969. 
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in the order of $5 million in a careful time-phased schedule (Seidel, 9). 
We submit that this is certainly not off by more than a factor of two, 
and further that the value of such a development, as we have tried to 
indicate, can be monumental in reshaping the whole field of education. 

Of course this must include adequate interdisciplinary staffing and 
evaluation and testing of all the components of experimental hardware, 
computer software, instructional strategy, and various selected subject 
matter. However, we submit again that the expenditure is but a minute 
fraction of the cost for other technological developments. Surely the 
goals are at least equally desirable. Also, given the 3% to 5% invest- 
ment in all of education, enough money should be available for such 
innovative efforts as CAI to permit, as Oettinger has proposed (2), a 
great deal of flexibility and diversity in approaches. It thus should 
be possible to solve the educational technological problems of using 
the computer properly in the entire field of instructional and educa- 
tional improvement, 

A Proposed Partnership*. Allocate the Unique Resources 
Among the Steps of a Systems Approach 

It seems clear that we are faced in education and training with (a) a 
commitment to unprecedented financial investment, (b) the requirement 
that this effort serve the public welfare, vis-a-vis the educational 
consumer, as opposed to a private corporate entity and its stockholders, 
and (c) the recognition and acceptance that the return on investment 
may take much longer for accurate calculation than do industry s 
profit and loss statements. All of this indicates that only the 
Federal government can adequately and properly provide the funding 
(and thereby influence the policy guidelines) for this type of 
partnership . 

Secondly, the techniques of industrial research and development 
(i.e., product development) are appropriate to contribute to an evolving 
CAI system. Various industries have the facilities and the tools for 
development and production (certainly for the necessary hardware and 
software components in CAI). Non-profit industry employs research 
personnel. The educational community is a unique partner in the sense 
that the local school system and its personnel aie going to be the focal 
point of the development, but they do not have the financial resources 
nor do they have more than a portion of the total personnel required to 
accomplish this innovation properly. Their partnership contribution 
consists in developing instructional content through providing subject- 
matter experts as part of an interdisciplinary team, and by providing, 
the demonstration and test facility within which the innovative experi- 
mentation and development can take place. 

Finally, in addition to the partnership structure, the developing 
CAI effort must include, as indicated earlier, a functional approach 
that embodies systems design and total systems development using the 
eight steps. 

To place the partnership within this approach, the initiation of 
policy and guidelines (statement of need ) would come from a central 
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government agency. (It is conceivable that in a somewhat different 
form, specific needs might arise from statements within the educational 
community.) These would be transformed into specific system objectives 
jointly by team members representing all these arms of the partnership. 

An interdisciplinary research and development effort (comprising the 
remaining steps except evaluation and feedback ) would be conducted 
with the lead role most probably taken by a non-profit (rather than 
profit-oriented) R§D corporation (see last two pages for reasoning). 

Local government and educational system personnel and facilities 
would provide administrative aid. Industry would fabricate the 
necessary hardware and software, and members of the three-way team 
would then attempt to implement the provisional system. At this stage, 
the work primarily would be accomplished by industry and the educational 
community with administrative aid provided by local government (Figure 2) . 

Functioning Model in the Systems Cycle 



IMPLEMENTATION 




Figure 2 



Considering the possibility of this joint venture as indicated, one 
more important link must be added. To provide the necessary objective 
evaluation and feedback in a coordinated manner, an independent fourth 
party is required. The form of the feedback information would be 
appropriate to the particular partner of the three-way team in order 
to make appropriate modifications to those aspects of the systems 
development process uniquely under its jurisdiction. For example, if 
modifications were required regarding statements of need, this could 
be provided to the policy and guideline process for the government. As 
the modifications were necessary in fabrication of equipment and soft- 
ware, it would be given in unique form to industry, and so forth. 

While this may be difficult to conceive of within a given country, 
it may not be far oft from what is currently proposed by the Organization 
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for Economic Cooperation and Development (OECD), an international 
body to establish policy and make apolitical, dispassionate and objec- 
tive evaluations of CAI studies. In the United States fragmented 
examples are beginning to appear. 

PROBLEMS TO SOLVE 

Problem of Interpreting the Value of Education 

Now we shall return and perhaps answer some of the rejoinders that 
may have developed to our proposal of a reasonable level of dollar 
investment. Let us consider the fact that the goal of a space program 
such as putting a man on the moon and retrieving him is quite specific. 

The parameters for guidance, for control of the entire system required 
to perform this feat, although large, are finite, measurable, quanti- 
fiable. A major problem for education is to identify all those factors 
that are pertinent to estimating requirements for specified achievement 
characteristics, and also for interpreting the value of that educational 
achievement. 

Another paramount problem that educational technology faces when 
contrasted to a neat and closed engineering technology such as the 
space program, is that the return on investment on education may not 
be visible for roughly an entire generation. The attainment of 
objectives in the space program, on the other hand, was relatively 
immediate, dramatic, quite tangible, and rapid. Our only rejoinder is 
that in educational technology one must extrapolate from immediate cri- 
terial effects of measured educational proficiency to ultimate criteria 
of general societal significance. Of necessity this means at the outset 
that we have clearly identified the parameters, our educational inputs 
and our educational outputs. 

We must not permit our experience with the ambiguities and vagaries 
of the traditional educational instructional system to force a premature 
and inappropriate evaluation of computerized instruction. True, there 
have been difficulties in measuring a "good” teacher, certainly in 
traditional instruction one of the most significant educational inputs 
(Froomkin, 10). But, as noted previously (Kopstein and Seidel, 1; 

Seidel, 9; Seidel, 11), in computer- administered instruction we can 
objectively document the dimensions of that system. Cost effectiveness 
can eventually be measured. The instructional agent's value (instead of 
the human teacher's) can be measured against its output (student achievement). 
The costs of the entire CAI system, input, transformation and output, can 
be evaluated and justified or not, on tangible and objective bases. 

What then are the implications for consideration of cost effective- 
ness studies in CAI? Granted that we may extrapolate to later generation 
returns, it certainly seems we must consider direct outputs of the 
instructional system if our evaluation is to have any substance at all. 
Standardized achievement units must be the criteria (Randall and 
Blaschke, 12). Harley (3, p. 52) in an insightful appraisal makes 
the point even stronger. "... we have been concerned with the cost 
per student taught (our input) and not the cost per student learned (our 
output); yet we know that what is taught and what is learned are not 
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synonymous." We have ignored therefore the vital "... cost of our 
scrap (non-operable units)— students who cannot function in today's 
society ..." and therefore our cost statements have fallen far short 
of the true costs of our education system. 

A host of other problems beyond the scope of this paper relate to 
establishing other measurable values of education. For instance, Suppes 
(13, p. 12) has recently called for "clearly stated normative principles" 
for dealing with the contradictions in modern philosophy of education. 
While not a direct concern of economics and CAI, certainly resolving 
the existing antonyms of educational philosophy (e.g., maximum freedom 
of choice versus development of a sense of discipline, content versus 
method, social adjustment versus maximum achievement) is essential for 
a meaningful evaluation of any educational innovation. This is par- 
ticularly critical (perhaps propaedeutic) for CAI which depends upon 
acceptance to a large degree of the model of individualized instruction. 
We believe that it is an extremely important problem to be resolved. 

Problems Exist in Establishing a 
Potential Partnership Mode l 

Another item is equally important in establishing that CAI is 
valuable, and when setting the necessary dollar figures for R§D invest- 
ment. That is, what kind of a workable framework can exist, within 
which this research, development and operational environment is 
appropriate? Can industry, government and the educational profession 
be combined to bring the effort to fruition? What policies must be 
incorporated to bring this about? How can resources, money, facilities, 
personnel, and time be most appropriately allocated? 

Are Goals Compatible? 

Before delineating the proposed prototype for the three-way partner- 
ship, let us examine potential problems that may arise from the combina- 
tion of industry, government and educational professions in this manner. 
One can ask whether the model of research, development, and operational 
utility for industry is compatible either with the educational profession 
or with that of the government. Interestingly, we already have at least 
one failure in an attempt to bring these three entities together. 

Project ARISTOTLE was intended to be a catalyst and continuing stimulant 
for educational innovation through these three arms, and it has been 
dropped for lack of support. Perhaps the answer lies in incompatible 
models. Industry in the United States exists to manufacture and sell 
at a profit. Its investment of 3% to 5% in research rests clearly in 
the belief that the return will occur in a relatively short period, 
perhaps two to three years. The premise becomes muddied when this 
industrial template is applied to education. We spoke earlier of the 
generation lag for evaluation purposes in educational products. 

Considering the educational model, the goals of education have never 
been defined in terms of profit and loss statements. Selling a product 
has never been part of the system. In fact, a frequent criticism in 
recent years has been that the intellectual aspects of education were 
being subverted for more specific and practical occupational training* 



It seems that the traditional roles of industry and the educational 
profession are sc antagonistic, or at least not overlapping, that 
throwing them together without creating a new compatible modal for R§D 
purposes in CAI is to create an anomaly doomed to failure at conception. 
Moreover, government and education are interested in welfare of the 
populace, education being— in this sense — a subset of the governmental 
function. Industry is concerned with product development and sales. 

But need these be different? (Galbraith said in The Affluent Society 
that the private sector could not handle this type of effort on its 
own, but why not a partnership?) 

"Government, Industry, and Education 1 1 
Are Not Monolithic Entities 

The so-called education ''industry" in CAI is not monolithic and 
not coordinated. It involves hardware manufacturers, book companies, 
and computer software houses at the very least. The companies are 
relatively independent with their own profit and loss statements, 
corporate policies, and so forth, and they all must be brought together 
in order to construct a meaningful CAI system. The educational community 
and the government are also made up of components with differing capa- 
bilities. "Educational community" consists of at least administrators, 
middle management in the person of assistant principals (or assistant 
headmasters) and the teachers on the one hand. On the other hand is 
the ultimate user, the student. Government is represented by central 
and local spheres. For funding, we require federal assistance and for 
participation in implementing CAI, we require local involvement. 

There may well be many objections to a proposed cooperative set of 
overlapping functional relationships among industry, government, and 
the educational community. Not the least of these is an abhorrence at 
attempting to apply the corporate model of R§D to education. But we ask 
the question: Why not? It has produced inventiveness, increased 

profits and viable, new products in the marketplace. Why not ask the 
same of education R§D? 

Will the corporate model apply? If education is to be considered as 
is industry, it is peculiar in that the financial resources are clearly 
not within the system per se, but must be provided by outside sources — 
the local, regional, and central governments. Recently, this has been 
called into question by those who would like to apply the corporate 
model (13); but the alternatives are not being readily accepted (they 
imply clear, behavioral definitions of educational objectives, etc.) by 
the educational community. 

This difficulty is epitomized in a statement by a noted special 
assistant on education who was in the former presidential administra- 
tion — "Perhaps the most traditional . . . , end the one (local 
community) most resistant to outside change has been the educational 
community" (14) . The purpose of the Elementary and Secondary Education 
Act of 1965, Title III program, is to provide federal grants directly 
to local schools "... for the very purpose of stimulating innova- 
tion and change in local educational patterns." The prevailing mood, 
however, has been deep-seated suspicion in the United States that 
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federal aid means federal control. Consequently, a serious question 
is: Even given sufficient funding resources, how can a workable coopera- 

tive model be developed to ensure (a) valid R§D involving the educational 
community, and (b) proper implementation in local systems. Part of the 
solution must of necessity involve commitment of dollar and "personnel 
resources toward large-scale R&V efforts in CAI and toward massive 
training and retraining in the educational community . 

Let us turn the question around and ask: Is education becoming an 

industry? Should it be? The problem would seem to be one of retaining 
the goals of intellectual expansion, freedom, and innovation, while 
adopting where feasible, a model of improvement in education drawing 
upon the techniques of industrial development. What also seems to be 
required at the very least are policy guidelines from the federal 
government to force a necessary workable structure. Assuming that the 
policy is appropriate to the task, this would aid not only industry, 
but the educational profession and the educational user — the student. 

A Vehicle for CAI Research, Development, 
and Implementation is Proposed 

A definite means by which the proposed partnership could proceed 
efficiently is the last item we will discuss. Within the United States, 
there are two such vehicles. We propose first the establishment of a 
national center for research and development on innovation in educational 
systems. (We compliment the Center for Educational Reseaich and Innova- 
tion that now exists in France.) We think it is necessary to establish 
such an institution. In fact, this has been proposed previously for 
the U.S. Navy Department, but was not funded by the Department of 
Defense. The Center’s prime function would be to coordinate the appli- 
cation of diverse scientific and technological principles in the solution 
of educational problems and, generally, to evolve educational technology 
to higher and higher levels. At any rate, this center would have 
closely tied regional satellites to carry out the translation of the 
results of the research and development into operational reality within 
local school systems. The fourth element, evaluative entity, described 
earlier, fits here. It would be this entity that would bring together 
the industry, central government funds, and local governmental and 
educational systems for reorienting personnel into a demonstration 
program. 

At the national center a training emphasis would be given to pre- 
and post-doctoral levels in order to develop increased national 
competence in instruction theory. The areas to be included might be 
illustrated by Computer Sciences, Behavioral Sciences, Applied 
Mathematics, in general what we might call educational- technological 
research, technical writing programs, and so forth. On the regional 
level, the regional centers would train for tasks relating to the 
implementation of CAI, that is orienting communities, training local 
officials, local administrators of programs, and so forth. It is 
important to make clear that the national R§D Center will not be the 
sole technical, or active research, installation. Research and 
development would be carried on at local installations to permit 
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flexibility in approaches and there would be technical feedback or input 
in both directions to upgrade the R§D status in general. A plan for 
coordinated activities among the regional and the national centers would 
be essential to avoid fragmentation and lost efforts. 

To accomplish testing and implementation there already exists a 
potential prototype of a fourth party in the United States. It is a 
nonprofit corporation (the Institute for Politics and Planning) which 
has the role of bringing together government, industry, and educational 
systems. This takes the form, for example, of establishing advanced 
learning centers in conjunction with the government, in this case not 
only the central government as the funding source but the local govern- 
mental groups as well as the local educational system participating as 
members of a testing and evaluation team. (Currently the profit-minded 
industry operates on a fixed-fee basis, and gets paid if, and only if, 
students succeed in reaching established achievement criteria.) 

This effort is somewhat premature with respect to CAI because it 
assumes that we are now able to accomplish CAI implementation on a 
large scale. Nevertheless, a nonprofit corporation may well be the 
focal point for making a partnership viable regarding all the activity 
phases, research, development, and implementation. In the nonprofit 
corporate model, goals tend to be more directly oriented toward the 
welfare of society than those of the profit maker who must put survival 
first. Alternatively, it may be possible to adjust the goals of the 
profit corporation. If the return on investment cycle were extended 
(to five or more years) through the provision of some type of government 
"insurance, " profit-making industry could play a role equivalent to the 
nonprofit entity. The "insurance" debt could be retired at the end of 
some agreed-upon period either out of the corporation’s assets (if no 
marketable product was delivered) or as a percentage of the profits 
derived from an implemented product. This at least represents one 
mechanism to consider. 

In any event, the partnership model would have to encompass all 
the facets of research, development, and implementation. Too frequently, 
an innovation is developed and implementation is given short shrift in 
the form of simply a written recommendation. To quote Mr. S. Clark 
Beise (Chairman, Executive Committee, Bank of America) in the report 
on innovation in education from the Committee for Economic Development, 
"One of the major problems inhibiting change in our present educational 
programs and processes is the lack of communication between educators, 
teachers, administrators, school boards, and the public." 1 He goes on 
to note that the statement in the report by the CED develops a program 
that should be accomplished for research in innovation but does not 
carry the recommendation through sufficiently "to the point of being 



1 Platt, a systems analyst, adds the dimension of unawareness 
" ... of the investment aspects of education ..." to the problem. 

He describes the public and private school system as a " . . . network 
with very few nodes sufficiently concentrated to allow one to enter and 
perturb the system." (15, p. 417) It seems our task, then, is to create 
the requisite nodes! 
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able to demonstrate their value to those who must be convinced that 
changes should be made. In order to disseminate information on recom- 
.nended changes effectively 3 there should he established a system of 
demonstration schools 9 reasonably available geographically 3 to show 
what can be done in general practice to implement and integrate the 
recommended improvements s within practical costs 3 into a rounded 
program. n (Italics added.) 

We trust also that this discussion has demonstrated that "CAI is 
worth doing," and that the approach proposed herein has suggested a 
reasonable framework for "doing it right." We are certain that alterna- 
tive proposals can and will be mf.de to deal with the complex problems 
of resource allocation for CAI research, development, and implementation. 
The rough sketch given in this paper can be viewed as no more than an 
opening wedge. However, we are equally certain that some such approach 
must be put forward and developed as a workable model if CAI is to live 
and fulfill its vast potential. 

Although we have been concerned with the field of education, this 
discussion generally applies to training and CAI also. To the extent 
that training can represent a more well defined subset of instructional 
processes (e.g., objectives and achievement measures) then the task 
of moving from research through development to operational CAI is 
made easier. But the steps are the same, the required commitment is 
firm, and the goals are still waiting to be achieved. 
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Footnotes 

1. The research reported herein was performed pursuant to a contract with 
the Office of Education, U. S. Department of Health, Education, and 
Welfare to the Learning Research and Development Center, University of 
Pittsburgh. Contractors undertaking such projects under Government 
sponsorship ere encouraged to express freely their professional Judg- 
ment in the conduct of the project. Points of view or opinions do not, 
therefore, necessarily represent official Office of Education position 
or policy. 

2. Requests for reprints should be sent to Wayne Donaldson, Department of 
Psychology, University of Pittsburgh, Pittsburgh, Pennsylvania, 15213. 

3. Throughout the paper d' is to be interpreted simply as the distance of 
the ROC curve above the chance diagonal as measured at the point of 
intersection of the ROC with th° negative diagonal. When the term is 
used it will always be clear, usually through pre-modification, whether 
it comes from recall or recognition procedures . For recognition studies 
this is the standard definition of d ' . For recall data it is equivalent 
to what Clarke, Birdsall and Tanner (1959) label J d~. In a later 
section of the paper d' is modified as being forced-choice, the 
definition being in the text. While the interpretation of d' may 
differ depending on whether the ROC is derived from recall or recog- 
nition data, it is Just that question of interpretation that Is being 
examined here and consequently ci' will be used throughout only as a 
descriptive statistic. 
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Figure 



d' as a function of triads for all groups in Experiment I. 
Also plotted is d' as a function of trials for the group 
receiving all-recall trials after Bernbach's (l9t>7) suggested 
"correction for guessing" has been applied to the data. 

d’ as a function of trials for Experiment II. Also plotted 
is d_' as corrected by Bernbach’s formula with (a) a constant 
guessing rate and (b) a trial-dependent, changing guessing 
rate derived from the confidence rating data. For purposes 
of comparison, the unccrrected d’ for the group receiving 
all-recall trials in Experiment I is also shown. 

The reciprocal of the probability correct, conditionalized on 
the confidence rating given by S_, as a function of confidence 
rating (Experiment II). 

Cumulative per cent frequency of use of different confidence 
ratings, with trials as the parameter (Experiment II). 

d' as a function of trials for the recall and recognition 
groups of Experiment III. Also plotted is d’ as a function 
of trials after Bernbach’s suggested "correction for 
guessing" has been applied to recall data. 



Figure 6 



Figure 7 



Forced-choice d' for the recall group of Experiments I, II 
and III and standard ROC d' for the recognition groups of 
Experiments I and III . 

Proportion of items called "correct" as a function of actual 
proportion of correct items (recall conditions from all 
experiments). Individual points are for separate trials. 
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